Industry News

Micron Introduces High Capacity High Bandwidth Memory (HBM) with Improved Performance to Enable Generative AI Innovation

Update time : 2023-08-02 12:01:30

Micron Technology Inc. today announced that the company has begun sampling the industry's first 8-layer stacked, 24GB capacity, second generation HBM3 memory with more than 1.2TB/s of bandwidth and a pin rate of more than 9.2Gb/s, which can deliver up to a 50 percent performance improvement over existing HBM3 solutions currently available in the market. With a 2.5x performance-per-watt increase compared to the previous generation, Micron's second-generation HBM3 products set new records for performance, capacity and energy efficiency metrics in critical artificial intelligence (AI) data centers, and will help the industry shorten the training time for large-scale language models, such as GPT-4 and higher, provide an efficient infrastructure for AI reasoning, and reduce total cost of ownership (TCO).

Micron introduced a High Bandwidth Memory (HBM) solution based on the industry's leading-edge 1β DRAM process node, which packages 24Gb DRAM die into industry-standard-sized 8-layer stacked modules. In addition, Micron's 12-layer stacked 36GB capacity product will begin sampling in the first quarter of 2024. Micron's HBM3 solution offers a 50 percent increase in capacity compared to other 8-layer stacked solutions currently on the market. The improved performance-to-power ratio and pin speeds of Micron's second-generation HBM3 products are critical to managing the extreme power demands of today's AI data centers. Micron has achieved significant improvements in energy efficiency through technology innovations such as doubling the number of silicon through vias (TSVs), increasing metal density by 5x to reduce thermal resistance, and designing more energy efficient datapaths compared to other HBM3 solutions in the industry.
Micron, a longtime storage leader in 2.5D/3D stacking and advanced packaging technologies, is honored to be a partner member of TSMC's 3DFabric Alliance to build the future of semiconductor and system innovation. Micron and TSMC have worked hand-in-hand during the development of the second-generation HBM3 products, laying the groundwork for the smooth introduction and integration of compute systems in AI and high-performance computing (HPC) design applications. TSMC has now received samples of Micron's second-generation HBM3 memory and is working closely with Micron on the next steps of evaluation and testing to help customers innovate their next-generation HPC applications.
Micron's second-generation HBM3 solution addresses the growing demand for multimodal, multi-trillion parameter AI models in the generative AI space. With a single module capacity of up to 24GB and a pin rate of more than 9.2Gb/s, the Micron HBM3 can reduce total cost of ownership by reducing training time for large language models by more than 30 percent. In addition, Micron HBM3 will trigger a significant increase in daily queries, resulting in more efficient utilization of trained models. With industry-leading performance per watt, Micron's second-generation HBM3 memory will effectively drive operational savings in the modern AI data center. In 10 million deployed graphics processing unit (GPU) use cases, a single HBM module can save 5W of power consumption, saving up to $550 million in operating expenses over five years.
Micron's second-generation HBM3 solutions are designed to deliver superior AI and high-performance computing solutions to our customers and the industry," said Praveen Vaidyanathan, vice president and general manager, Compute Products Business Group, Compute and Networking Division, Micron. A key consideration for us was the ease of integration of the product on our customers' platforms. Micron HBM3 features fully programmable Memory Built-In Self-Test (MBIST), which runs at full specification pin speeds, enabling Micron to provide customers with enhanced test capabilities, enabling efficient collaboration and helping customers reduce time-to-market."
Ian Buck, vice president of hyperscale and high-performance computing at NVIDIA, said, "At the heart of generative AI is accelerated computation, where the high bandwidth of HBM is critical and leads to superior power efficiency. We have a long-standing relationship with Micron across a number of product areas and are very much looking forward to continuing to work with Micron on the second generation of HBM3 products to accelerate AI innovation."
The second-generation HBM3 is another milestone in Micron's leading-edge technology. Micron previously announced a 96GB capacity DDR5 module for high-capacity server solutions based on a 1-alpha (1-alpha) node with a 24Gb single DRAM die; now it is launching a 24GB capacity HBM3 product based on a 1β node with a 24Gb single DRAM die, and is planning to launch a 128GB capacity DBM3 product in the first half of 2024 based on a 1β node with a 32Gb single DRAM DDR5 modules with 128GB capacity based on 1β node, 32Gb single DRAM die. These new products demonstrate Micron's cutting-edge technology innovation in the AI server space.

Previous : Infineon High Voltage Superjunction MOSFET Series Adds Industrial and Vehicle Grade Devices for Static Switching Applications

Next : Dutch semiconductor equipment maker ASM: Q2 new orders down by nearly half on weak demand

How many chips does a car need?

Sep .19.2024

Automotive chips can be divided into four types according to their functions: control (MCU and AI chips), power, sensors, and others (such as memory). The market is monopolized by international giants. The automotive chips people often talk about refer to

Position and Function of Main Automotive Sensors

Sep .18.2024

The function of the air flow sensor is to convert the amount of air inhaled into the engine into an electrical signal and provide it to the electronic control unit (ECU). It is the main basis for determining the basic fuel injection volume. Vane type: The

Chip: The increasingly intelligent electronic brain

Sep .14.2024

In this era of rapid technological development, we often marvel at how mobile phones can run various application software smoothly, how online classes can be free of lag and achieve zero latency, and how the functions of electronic devices are becoming mo

LDA100 Optocoupler: Outstanding Performance, Wide Applications

Sep .13.2024

In terms of characteristics, LDA100 is outstanding. It offers AC and DC input versions for optional selection, enabling it to work stably in different power supply environments. The small 6-pin DIP package not only saves space but also facilitates install

About Us

About Us What we do Quality People

Products

CHIP/IC Active devices Passive device Components OTHER

China electronic components supplier----Wah Luen Shing Electronic.----www.wlschip.com

Leave a message